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APPLICATIONS  OF  SEMI-REGENERATIVE  THEORY  TO  COMPUTATIONS 
OF  STATIONARY  DISTRIBUTIONS  OF  MARKOV  CHAINS* 


by 

W.  K.  Grassnann  and  Michael  I.  Taksar 


1.  Introduction 

■  Arguments  from  Regenerative  Theory  have  been  used  by  a  nvunber  of 

authors  to  solve  equilibrium  equations  in  queueing  problems.  In  parti¬ 
cular,  these  arguments  are  prominent  in  the  matrix-geometrio  solution 
pioneered  by  Neuts  [1981] ,  but  they  are  also  used  by  Kleiprock  [l9T5l  in 
order  to  analyze  the  GI/G/m  queue  and  by  Grassmann  and  Chaudhry  [1982I . 
Since  there  is  a  number  of  applications  for  these  methods,  it  seems 
appropriate  to  investigate  them  in  further  detail. 

>In  this  paper  we  use  Semi-Regenerative  Theory,  which  is  a  general¬ 
ization  and  sophistication  of  Regenerative  Theory.  We  believe  that  this 
is  the  first  paper  which  uses  Semi -Regenerative  Theory  for  developing 
numerical  (nonsiraulation)  algorithms  to  find  the  steady-state  distribu¬ 
tion  of  a  Markov  chain.  The  algorithm  obtained  is  a  modification  of  the 
Gauss-Jordan  method,  in  which  all  the  elements  used  in  con5>utations  are 


always  nonnegative,  which  makes  the  algorithm  numerically  stable. 


To  apply  the  theory  in  question  to  a  given  Markov  chain  = 

0,1,...)  we  must  represent  the  latter  as  a  semi-regenerative  process. 
To  this  end  we  consider  a  subset  D  of  the  state  space  of  Y^^  and  we 
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8017867,  at  the  Institute  for  Mathematical  Studies  in  the  Social  Sci¬ 
ences  at  Stanford  University. 
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record  successive  visits  of  the  chain  to  this  subset.  Let  Tjj, 

n  =  1,2,...,  be  the  tine  of  n-th  visit  to  D  and  Xn  position 

of  the  chain  at  Then  the  process  ^  Markov  Renewal 

process  (see  Qinlar  [l975l  Chapter  10)  and  Y^^  is  a  semi -regenerative 
process  with  being  the  seiri-regenerative  epochs  and  (Xjj,Tjj)  being 

the  iirbedded  Markov  Renewal  process. 

The  latter  ireans  that  if  we  consider  the  "cycles" 

^n  ^  ^^T  ’’^T  +1’'‘*’^T  -1^ 

n-1  n-1  n 

then  the  conditional  distribution  of  the  sequence  (Aj^,Ajj+i  , . . . )  given 
the  past  of  the  process  up  to  depends  only  on  Xn-qi 

Ajj  are  conditionally  independent  given  (X*,T’). 

The  analysis  of  the  behavior  of  Y.  froir  one  seipi-regenerative 
epoch  to  another  produces  the  min  relation  between  the  steady- state 
probabilites  that  is  used  for  developing  the  algorithm. 

2 .  Proof  of  the  Main  Result 

We  consider  an  irreducible  aperiodic  positive  recurrent  Markov 
chain  Yp  with  a  state  space  E  =  l0,l,2 , . . .} .  It  is  known  (see 
inlar  [l975l  >  Chapter  8)  that  such  a  chain  reaches  a  steady  state, 
i.e. , 

^i^^n  "  ^  . 

or  in  a  shorter  version 

Y  =>  Y 
n 
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where  Y  is  an  integer-valued  random  variable  with  distribution 
(Pq, ?!»•••)•  Here  P^{*}  £  P{»  )Yq  =  i} .  The  notation  must  be 

understood  in  a  similar  way.  Let 

D  =  {0,1,2,...,  d  -  1} 

and 

(2.1)  T  =  min  (m  >  0:  Y  £  D) 

m 

Put 


(2,2) 


'kl’  •  'i,'  j  “■>  ‘‘m- 

m=0 


Theorem;  Let  (pQ,...,Pj^)  be  the  steady-state  distribution  of 


the  Markov  chain  Y^^.  Then 


(2.3) 


Pi  ■ 

^  k=0 


Proof:  Consider  the  process  =  Yj^j,  where  [tl  is  the 
integer  part  of  t.  Let 


Then  for  each  n  >  1  we  have 

(a)  Tjj  is  a  stopping  time  for  Z. 

(b)  Xn  is  determined  by  (z  ;  u  <  T  } 

(c)  for  each  function  f  defined  on  E"*  and  each 

t^<t2<***<  tjjj 


Ei{f(ZT  +t  ^  "  V 

n  1  n  m 


E,{f{Z  )}  on  the  set  {X  =  j} 

>1  ^  ^ 

1  m 


Really,  (a)  follows  from  the  definition  of  (b)  is  a  consequence  of 

(2.4)  and  (c)  is  due  to  the  strong  Markov  property  of 

In  particular  X^  is  a  Markov  chain  in  the  state  space 
D  =  tO,l,...,  d  -  li  .  This  chain  is  irreducible  because  the  original 
chain  Y^^  is  irreducible.  Let  v . ,  i  =  1,2,...,  d  -  1  be  the  (unique) 
invariant  distribution  for  Y^^.  Let 


m(i)  =  E.lT) 


and 


K^(j,i)  =  Pj{Z^  =  i,  T  >  t)  H  PjYj^j  =  i;  Y^  D,  for  k  <  T) 
According  to  Cinlar  [l975l ,  Chapter  10,  Theorem  (6.12) 

OP 

(2.5)  lim  P  (Z.  =  j)  =  (  },■  m(i)v  )~^  I  vJk  (k,j)dt  . 

t-^  ^  ^  /leo  k£D  *^0  ^ 

We  must  mention  that  (2.5)  was  proved  in  yinlar  [l975l  under  the 


assumption  that  (X,T)  is  a  positive  recurrent,  irreducible  and 
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aperiodic  Markov  renewal  process.  In  our  case,  (X,T)  is  irreducible 
and  positive  recurrent  due  to  similar  properties  of  I>ut  (X,T)  is 

not  aperiodic.  The  random  variables  are  always  integer  valued. 

Since  Yn  aperiodic  (as  a  discrete  Markov  chain)  then  (X,T)  is 
periodic  with  period  one.  However,  the  proof  of  (2.5)  in  Qinlar  [l975l 
goes  through  without  change  for  the  process  Z  such  that  =  Zj^j 
and  such  that  the  period  of  the  imbedded  Markov  Renewal  process  is  1. 
The  function  K^{k,j)  is  piecewise  constant,  namely 
hence  (using  Fubini*s  theorem  below) 

®  00 

(2.6)  JK  (k,j)dt  =  z  K  (it.j) 

0  ^  m=0  " 

00 

'  J ’’A  -  J-  ’’ ' 

m=0 

00 

=  I  E.{1,(Y  )1»^  } 
k  J  m  T>m 

m=0 

On  the  other  hand 


^  =  I  ^k^^m  =  J.  T  >  m) 


m=0 


■  -  ’'y' 


(2.7) 


lira  P.iZ^  =  j) 
t-  "  ^ 


lira  P. (Z  ,  , 


J)  =  lira  P  (Y  =  J)  =  p 


Combining  (2.5),  (2.6)  and  (2.7),  we  have 


(2.8) 


PJ  =  I  I  V  m(A)j-^  I  V  v^^^ 
JtGD  jeO 


Let  us  apply  (2.8)  for  J  £  D.  It  is  easy  to  see  that  if 


k,j  £  D  then  , 
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f  0  ^ 

1  1  if  k  =  J 


Therefore,  for  J  G  D  formula  (2.8)  becomes 

(2.9)  p,  = - - 

I  V  m()i) 
siSD  ^ 

Now,  substituting  (2.9)  into  (2.8),  we  get  (2.3). 

The  next  proposition  describes  the  relations  between  for 

kj 

different  k,J  and  d.  Its  significance  will  become  clear  later  when 
we  develop  a  numerical  algorithm. 

Proposition:  Let  be  given  by  (2.2),  and  let  p^  ,  be  the 

kj  J 

transition  probabilities  of  the  MarKov  chain  Then 


(2.10) 

ko 

kJ 

do  kd 

(2.11) 

kd 

"  ^kd  ^ 

J>d 

(d) 

Jd  ^ko 

(2.12) 


kj  kj  k« 


Proof: 

l)  Let  D  consist  of  the  first  d  points  from  0  to  d  -  1 
and  let  T  be  defined  by  (2.l).  Let  r  be  the  first  hitting  time  of 
the  point  d.  Then  we  can  write 
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(2.13) 


Tat-1  T-1 

=  V  ^  ^  V  ^ 

n=0  *  n=TAT  J 


The  first  term  In  the  right-hand  side  of  (2.13)  is  obviously  equal  to 

Next,  we  must  notice  that  if  t  >  T  then  the  second  expres- 
icj 

sion  in  the  right-hand  side  of  (2.13)  equals  zero.  On  the  set  t  <  T 
we  can  use  the  strong  Markov  property  to  obtain 


(2.1U) 


T) 


<  T) 


Substituting  (2.ll<)  into  (2.13),  we  get 


(2.15) 


^kj  '^kj  dj 


Now,  iterate  (2.15),  namely,  put  k  =  d  in  (2.15)  and  substitute  the 
value  of  right-hand  side  of  (2.15).  We  get 


kj  kj  k  ‘  dj  dj  d 


(2.16) 

Iterating  (2.15)  n  times,  we  get 


(2.17) 


<  T)  <  T})“ 


IsJ  kJ 


m=0 


dJ 


+  (p^{t  <  T})"v^J^l 


If  P^{t  <  T}  =  1  then,  using  the  strong  Markov  property,  we  get  that 
Pj{T  <  *}  =0,  which  contradicts  the  assumption  that  is 
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irreducible.  Therefore,  <  T)  <  1  and  we  can  pass  to  the  limit  in 

(2.17)  as  n  + 


(2.18) 


<  T)j' 


Using  the  strong  Markov  property  once  more,  we  see  that  if  k  ^  d,  then 


(2.19) 


T-1 


I  P  {x  <  T}[P  {x  <  T}]”^  =  I  P.  {  I  >  m) 


m=0 


k*-  ^ 

m=0  t=0 


T-1 

^  t=0  * 


Combining  (2.l8)  and  (2.19),  we  get  (2.10). 

2)  To  prove  (2.11),  write  (below  we  use  the  fact  that  T  =  1  on 
the  set  £  D) 


(2.20) 


'kd’  =  V  *  Vh(h>h>i‘ 

ra=l 


m=2 

If  Yj^  =  d  then  obviously  T  >  1;  therefore,  the  first  term  in 
the  right-hand  side  of  (2.20)  equals 

=^‘id<h»  -Pkd 

Due  to  the  Markov  property,  the  conditional  expectation  in  the 


second  term  in  the  right-hand  side  of  (2 .20)  equals 
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1  m=l  1 


and  hence  (2.20)  becomes 


'''  ^  ~  Pkd  '*■  V'^Yj^d 


kd 


r  (d' 


3)  Formula  (2.12)  generaliz*  ^O),  its  proof  and  the  ideas 

behind  the  proof  are  similar  to  those  of  (2.10),  but  require  more  compu¬ 
tations,  and  we  omit  them  here. 


3.  An  Algorthm  to  Find  Equilibrium  Probabilities 

In  this  section,  we  present  an  algorithm  to  find  the  equilibrium 
probabilities  p^,  which  will  then  be  interpreted  in  terms  of  the 

This  interpretation  is  interesting  for  its  own  sake,  but  it  can 
also  be  exploited  in  a  number  of  ways  as  will  be  shown. 

The  equilibrium  probabilities  Pj^  are  given  by  the  steady  state 
equations,  that  is 


0  = 


N 

I 

i=0 


PiPiJ 


-Pj 


=  Z  p,  (p. ,  -  *•  J 


i=0 


ij 


ij  ' 


J  =  0,1,. 


.,N 


Here  6^^  is  1  if  i  =  J  and  zero  otherwise.  To  find  the  Pj^^  we 
proceed  as  follows.  We  solve  the  Nth  equation  for  pjj,  and  eliminate 
Pjj  from  all  other  equations.  Then,  we  solve  equation  N  -  1  for  Pu_i» 
and  eliminate  P[j_i  from  all  other  equations,  except  from  equation 


-10- 


N.  We  continue  this  way  until  we  solve  the  first  equation  for  p^.  In 
other  words,  we  always  use  the  diagonal  element  as  pivot,  and  we  apply 
Gaussian  elemination,  starting  with  equation  •  N  and  ending  with  equa¬ 
tion  1.  If  are  the  values  obtained  before  solving  for  p_ ,  one 

ij  " 

has 


and 


(3.1) 


(3.2) 

(3.3) 


n 

1 

c 

a. 

-  - 

’in 

^nn 

n-1 

n 

iJ 

=  a.  .  + 

n-1 

n 

'iJ 

=  a . 

,  0  <  i  <  n 


0  <  J  <  n 


This  method  can  onij  be  applied  if  all  >  0.  This  is  where  the 

nn 

theory  presented  earlier  is  helpful.  Clearly,  the  elimination  procedure 
gives 


.1-1 


I  P.a", 

i=0^ 


J  >  n  . 


It  is  also  clear  that  the 
h  >  J.  On  the  other  hand. 


a”  are  uniquely  determined  by  the 
iJ 

we  have  equation  (2.3),  which  in^ilies 


J-1 


(J) 


^  Pi^iJ 

i=0 
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Here,  the 
Hence 


are  similarly  determined  uniquely  hy 


''ik’ 


k  >  J. 


(3.U) 


J  >  n  . 


This  implies  that  0  <  a”j  <  *,  J  >  n.  This,  together  with  (3.1)  im¬ 
plies  that  a”  cannot  be  equal  zero.  We  can  even  s€iy  more.  Indeed, 
nn 

one  has 


(3.5) 


nn 


n-1 

=  -  I  a 


J=0 


n 

nj 


To  prove  (3.5),  we  show  by  coii5)lete  induction  that 


(3.6)  I  a"  =  0 

J=0 

Clearly,  this  equation  holds  for  n  =  N.  We  now  prove  (3.6)  for  n-1. 
Because  of  (3.3),  we  only  need  to  consider  the  case  where  J  <  n. 
Equations  (3.1)  and  (3.2)  then  give 


n-1 

I  a 


J=0 


n-1 

ij 


n-1 


J=0 


n  n 
a  ,a. 
nJ  in 

n 

a 

nn 


•) 


=  Z  a 

J=0 


n 

iJ 


n-1 

l  a 


J=0 


n 

iJ 


n 

a 

nn 


n-1 

=  I 

J=0 


+  a. 


in 


J=0 


iJ 


=  0  . 


This  proves  (3.6),  and  with  it  (3.5) 
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At  this  point,  it  should  become  apparent  that  the  elimination 

suggest  by  (3.1)  to  (3.3)  can  be  done  without  ever  using  subtractions 

or  negative  numbers.  We  merely  need  to  replace  a”  in  (3.1)  by  (3.5), 

nn 

giving 


(3.T) 


n-1 


a. 

in 


n 

a. 

in 


n-1 

I  a", 

J=0 


0  <  i  <  n 


and  omit  (3.2)  for  the  case  that  i  =  J.  Equations  (3.2)  and  (3.7) 
involve  then  only  a^j  with  i  *  j,  and  these  will  always  remain  posi¬ 
tive,  given  they  were  positive  to  start  with. 

Because  of  (3.**),  the  elimination  described  by  (3.2)  and  (3.7) 
gives  for  n  =  0 


(3.8) 


0  (1) 

^  ^  ^  ’  J  =  1.2 . N  . 

vJ  i=0  ^  i=0  ^ 


From  (2.12),  we  find 


(3.9) 


Oj  OJ  Ok  \j 


Relation  (3.9)  allows  one  to  find  vl^^  recursively,  once  ^  is 

OJ  ^ 

given.  Once  all  are  found,  Pq  becomes 


( 3.10) 


(1)  .  „ 

Pj  =  PqVqj  ,  J  >  0 
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With  this,  all  Pj  are  found,  and  ve  can  now  count  how  many  operations 

are  needed  to  find  all  pj,  J  =  0,1,...,N. 

We  note  that  for  each  n,  ,  has  to  be  calculated,  which  gives 

nj 

n  operations,  eqioation  (3.7)  must  be  applied  n  times,  requiring 
another  n  operations,  and  equation  (3.2)  is  applied  n^  times,  giving 
another  2n^  operations.  Since  n  varies  from  1  to  N,  this  gives 
(see  also  E.  Isaacson,  H.  B.  Keller  (1966I ) 


N  p  N 

I  2(n  *  n)  =  2  I  n(n  +  l)  = 

n=l  n=l 


2N(N  -*•  1)(N  +  2) 
3 


operations.  The  number  of  operations  is  thus  of  order  2n3/3, 


Frequently,  there  exist  g,h  such  that  =  0  for 

J  <  i  -  g  and  J  >  i  +  h.  The  a?  inherit  this  property  as  can 

*  J 

be  seen  by  coji5)lete  induction.  To  find  the  demoninator  of  (3.7)  then 
requires  only  min(n,g)  additions.  Once  the  denominator  is  found, 
(3.7)  needs  only  be  applied  min(n,h)  times,  which  gives  a  total  of 
rain(n,h)  multiplications.  (3.2)  similarly  requires  a  total  of  2 
min(n,h)  •  min(n,g)  operations.  Since  n  runs  from  1  to  N,  this 
gives  order  2Nhg  operations,  which  is  considerably  less  than  2N3/3 
equations  as  long  as  h  or  g  is  small  compared  to  N.  The  number  of 


operations  to  do  (3.9)  and  (3.10)  is  always  relatively  insignificant. 

Sometimes,  the  v^'^^  J  ^  1  and  J  ^  d  are  needed  as  well.  They 
are  easily  found  recursively  from  the  ^ »  using  equation  (2.12). 
Moreover,  by  substituting  v^^^  in  (2.3)  from  (2.11),  we  find  the 


following  interesting  equation 


-lU- 


(3.11) 


Thus,  once  the  are  found,  we  have  J  equations,  and  (j  -  l) 

independent  equations,  which  allow  one  to  find  Pj,  J  =  0,1,...,  d  -  1, 
except  for  a  factor.  Applications  of  (3.11)  will  be  given  in  the  next 
section. 


4.  Applications 

To  illustrate  the  method,  we  now  consider  a  number  of  special 
cases  and  discuss  some  numerical  results.  Some  of  the  examples  dis¬ 
cussed  deal  with  continuous-time  Markov  processes.  This  is  no  major 
problem  because  all  the  methods  carry  over  to  continuous-time  Markov 
processes,  provided  p,  ,  i  ^  J  is  replaced  by  the  rate  .  Also, 

1 J 

the  v^*?^  can  be  interpreted  similarly.  The  diagonal  elements,  that 
is,  the  Pj^j^  -  1,  respectively,  the  a^j^,  are  irrelevant.  We  can  set 
these  elements  equal  to  0. 

In  most  cases  considered  here,  pj^j  (respectively  a^j )  are  zero 
for  J  >  i  +  h  and  J  <  i  -  g.  In  this  case,  one  finds  from  (3.2), 
(3.4)  and  (3.T) 


i  <  n  -  h 


(4.1) 


in 

n-1 

T  a 
J>0,n-g  "’J 


0  ,n-h^  i<n 
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n  ^  n  (n)  0,  n-h<i<n 

ij  ®^nj^in  0,  n-g<J<n, 

otherwise 
ij 

It  can  now  immediately  be  seen  that  in  the  case  of  a  birth-death 
process,  where  a^^  and  i  ]^  =  Wj.  a^j  =  0,  J  >  i  +  1  or 

J  <  i  -  1,  (^.2)  will  not  be  used  at  all,  and  (4.1)  becomes 

^(n)  ^  ^n-1 

n-l,n 

Relation  (3.8)  then  leads  to  the  well-known  relationship 

pj  ■  ^  ■ 

This  result  gives  a  nice  check  for  our  derivations.  We  also  note  that 
our  method  does  not  pick  the  (less  efficient)  simple  recursive  solution 
of  the  steady  state  equations. 

Next,  consider  the  E^/M/1  queue,  with  the  additional  restriction 
that  the  number  of  "phases"  in  the  system  is  restricted  to  N.  In  this 
case,  a^j  =  u  for  J  =  i  -  2 ,  i  =  2 ,3 , . . . ,N  and  a  =  2X  for 
J  =  i'*’l»  i=0,l,...,N-l.  All  other  a^j ,  i  ^  J  are  zero. 

Since  h  =  1,  (4.1)  is  only  applied  for  i  =  n  -  1,  that  is 


(4.2) 


n-1 


(4.3) 


n-l,n 


2X 


a  ,+  P 
n,n-l 


Equation  (4.2)  is  similarly  applied  only  for  i=n-l,  J  =n-2. 


which  gives 
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(*♦.*»)  +  a" 

^n-l,n-2  ^n-l,n-2  ®'n,n-2^n-l,n  ''^n-l,n 

Equations  (14.3)  and  (U.U)  can  be  combined  to  give 

(n) 


2X 


+  P  ~  1  +  V 
n,n+l  n,n+l 


ii 

U 

nSTT  • 


Moreover,  it  is  easily  verified  that  „  =  2A/p,  This  means  that 

N-1  ,N 

is  really  the  beginning  of  a  continued  fraction 

n-l,n 


(N-1) 
N-2 ,N-1 


u 


1  + 


2X 


(N-2) 

N-3,N-2 


2X 

11 


1  +  - 


2X 

V 


1  + 


2X 


and  so  on.  In  Table  1,  the  v^”,  are  listed  for  N  =  10, 

n-l,n 


.(n) 


li  =  1  and  X  =  0.5  and  1.  It  is  clearly  seen  that  the  v^_^ 
converge  rather  quickly.  Then,  due  to  (3.8), 

(n) 


Pn  = 


n-l,n^n-l 


Of  course,  our  algorithm  can  solve  much  more  complex  problems  than 
the  ones  presented  above.  To  show  this,  we  calculated  the  stationary 
probabilities  for  a  number  of  cases.  These  results  are  given  in  Table 


I 


2 
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Table  1: 

The  'til® 

n-l,n 

E2/M/I  queue 

X  =  0.5 

X  =  1 

n 

n-l,n 

n-l,n 

9 

1 

2 

8 

0.5 

0.6667 

7 

0.6667 

1.2 

6 

0.6 

0.9090 

5 

0.625 

I.OU76 

1* 

O.615U 

0.9767 

3 

0.6190 

1.0118 

2 

0.6176 

0.99^1 

1 

0.6182 

1.0029 

Table  2: 

Coii5>utational 

results 

N 

h 

g 

Execution 

Time* 

Average 

Difference 

Largest 

Difference 

Largest  Rel. 
Difference 

100 

10 

10 

1.6 

2  E  -  10 

5  E  -  10 

7  E  -  8 

1000 

10 

10 

15 

5  E  -  11 

2  E  -  10 

2  E  -  7 

100 

20 

10 

2.7 

E  -  10 

5  E  -  9 

7  E  -  8 

100 

10 

20 

2.7 

2  E  -  10 

2  E  -  9 

7  E  -  8 
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In  each  case,  the  a^^j  where  generated  randomly,  using  a  uniform  distri¬ 
bution  between  zero  and  1.  To  find  out  how  accurate  the  results  are,  we 
solved  each  problem  twice,  except  that  in  the  second  run,  state  0 
became  state  N,  state  1  became  state  M  -  1  and  so  on.  Absolute  and 
relative  deviations  between  these  two  methods  are  also  given  in  Table 
2.  The  execution  time  is  the  time  per  run  in  cpu  seconds. 

Applications  are  not  limited  to  numerical  investigations  as  the 
following  exan^les  indicate. 

Kleinrock  (19751  Investigated  the  imbedded  Markov  process  of  the 
GI/M/m  queue  as  follows.  He  defined  ^  to  be  the  expected  number 
of  visits  of  state  d  between  two  visits  of  state  d  -  1.  In  the 
process  in  question,  it  is  impossible  to  visit  state  d  from  the  states 
d-2,  d-3,...,  which  implies  that  is  also  equal  to  the  number  of 

visits  to  state  d  during  a  stay  among  the  states  d,  d+1,  d+2,.... 
In  symbols,  h  =  1,  and  consequently 

«  - 

d  ■  '^d-l.d  • 

Kleinrock  proves  now  that 


=  o 


d-1 


''d-1 


a  fact  that  immediately  follows  from  (U.l)  with  h  =  1.  Kleinrock 

argues  further  that  independent  of  d  as  soon  as 

p.  =  p...  ,,  h  >  0  is  independent  of  J.  This  also  follows  easily 

from  our  derivation,  because  determined  by  the 

d-l,d 

P14.V  4  •  J  ^  More  con^)lex  systems  than  Klelnrock's  can  be  analyzed  in 


i 
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a  similar  way.  In  particular,  our  methods  can  give  new  Insights  Into 
the  matrix-geometric  solution  of  Neuts  I1981I  as  the  reader  may  verify. 

Next,  we  present  two  applications  of  the  equation  (3.11}.  The 
first  application  concerns  a  system  with  a  huge  number  of  states,  where 
the  probabilllty  mass  is  concentrated  in  a  few  states  only,  say  in  the 
states  0,1,2,...,  d  -  1.  Such  systems  are  encountered  frequently  in 


connection  with  queueing  networks  at  a  low  traffic  intensity.  In  such 

systems,  we  can  obtain  the  vl*?^  simulation,  and  caluclate  the  pj. 

kj  " 

J  <  d  from  (3.11),  and  the  p^,  J  >  d  from  (2.3). 

The  second  application  of  (3.11)  concerns  a  group  of  Markov  pro¬ 


cesses,  in  which  the  Pij.  J  ^  d  are  identical,  but  the  Pij »  J  ^  d  ^re 
not.  Since  the  v|j^  depend  only  on  p^j ,  J  >  d,  this  means  that  the 
Vfj^  are  identical  for  all  these  problems,  and  need  not  be  recalculated. 
For  this  case,  (3.11)  provides  a  convenient  way  to  solve  such  problems. 


This  is  of  importance  in  the  case  of  sensitivity  analysis. 


5.  Conclusions 

It  has  been  shown  (Grassmann  [1983] )  that  algorithms,  dealing  with 
non-negative  elements  only,  and  containing  no  subtractions,  are  extreme¬ 
ly  resistant  to  rounding  errors.  In  this  paper,  we  derived  such  an 
algorithm.  The  algorithm  combines  elements  from  regenerative  theory 
with  algebraic  methods.  The  algorithm  is  still  effective  for  problems 
with  a  size  of  N  =  1000,  a  range  where  normal  Gaussian  elimination 
tends  to  fail  because  of  the  accumulation  of  rounding  errors.  The 
algorithm  also  lends  Itself  to  exploit  band-structures  which  are  preva¬ 
lent  in  many  queueing  problems. 
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Because  of  the  interpretation  of  some  key  elements  of  the  algo¬ 
rithm,  using  regenerative  theory,  the  algorithm  can  also  make  theoreti¬ 
cal  contributions.  In  particular,  it  makes  an  argument  of  Kleinrock 
more  transparent.  We  also  mentioned  that  it  can  be  used  in  connection 
with  the  matrix-geometric  solutions  of  Neuts.  Finally,  the  regenerative 
underpinnings  of  our  algorithm  allow  one  to  combine  analytical  and 
sinulation  methods. 
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